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What is claimed is: 



1 . A method of determining a revision of a test suite of a model-based 
diagnostic testing system comprising: 

evaluating a diagnostic^efficacy of the test suite using a probability of a 
diagnosis. 

2. The method of Claim 1 , wherein the evaluation comprises: 
suggesting a test to add to|he test suite to adjust an overall test coverage of the 

test suite. 

3. The method of Claim 2\ wherein suggesting a test comprises: 

I 

creating a simulation database of the test suite; 

determining a probability of alcorrect diagnosis and a probability of an incorrect 
diagnosis for the test suite using the database; and 

creating a list of suggested testsMrom the determined probabilities. 

4. The method of Claim 3, wherein each suggested test on the list comprises 
a test coverage. I 

5. The method of Claim 3, wherein the evaluation further comprises: 
identifying a test to delete from the\test suite that comprises: 
determining a probability of a correct diagnosis for a modified test suite using 

the database, the modified test suite havingia selected test removed from the test suite; 
computing an efficacy value associated with the selected test using the 



determined probabilities of a correct diagnosis for the test suite and the modified test 
suite; and I 

generating a list of deletable tests and associated efficacy values. 



6. The method of Claim 1, wherein thelevaluation comprises: 

identifying a test to delete from the test suite, the deletable test having a minimal 
effect on an overall diagnostic efficacy of the tesflsuite. 

7. The method of Claim 6, wherein identiwing a test comprises: 
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creating a simulatic&i database of the test suite; 

determining a probability of a correct diagnosis for the test suite using the 
database; u 

determining a probability of a correct diagnosis for a modified test suite using 
the database, wherein the modified test suite is the test suite having a selected test 
removed; 

computing an efficacy value for the modified test suite using the determined 
probabilities; and 

generating a list of deleta&le tests using the computed efficacy values. 

8. The method of Claims, wherein determining a probability for a modified 
test suite is repeated for a plurality of modified test suites, each modified test suite of 
the plurality being the test suite having a different selected test removed. 

9. The method of Claim 8, wherein the selected test associated with the 
modified test suite having a low computed efficacy value relative to other modified 
test suites is the deletable test. 

10. The method of Claim 7, wherein evaluating further comprises: 
suggesting a test to add to the testtsuite to adjust an overall test coverage of the 

test suite that comprises: \ 

determining a probability of an incorrect diagnosis for the test suite using the 
database; and \\ 

creating a list of tests to add from the|determined correct and incorrect 
probabilities for the test suite. 

11. A method of evaluating a diagnostic efficacy of a test suite of a model- 
based diagnostic testing system comprising: | 

creating a simulation database of the testTsuite; 

determining a probability of one or both of a correct diagnosis and an incorrect 



diagnosis for the test suite using the database; and 

using the determined probability to evaluate the test suite. 
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12. The method of ffilaim 1 1, wherein creating a simulation database 
comprises: \\ 

simulating an application of the test suite to a device under test, the device under 
test comprising one or more components; and 

recording a probable resu|t of the application in the simulation database, the 
simulation database being represented by a table having a plurality of columns and a 
plurality of rows, the plurality oficolumns comprising a component pattern, a test 
result pattern, and a number of occurrences, 

wherein the component pattern encodes which component is good or bad, each 
component of the device under testVbeing represented by a unique position number 
within the component pattern, \ 

wherein the test result pattern encodes which of the tests of the test suite failed 
or passed, each test in the test suite bling represented by a unique position within the 
test result pattern, \ 

wherein the number of occurrences represents a number of times that a given 
combination of the component pattern and the test result pattern occurred during a 
simulation, the number of occurrences being an integer greater than or equal to zero, 
and 

wherein each row of the plurality of i*pws corresponds to a different unique 
pattern of good and bad components. 

13. The method of Claim 12, whereiri\determining a probability of one or both 
of a correct diagnosis and an incorrect diagnosis comprises: 

copying to a database copy only those rows of the created simulation database 
with only one bad component in the componentWttern column, all other components 
in the respective row being good; ft 

sorting the database copy based on the test nattern column, such that the rows 
with a given test pattern are adjacent to one another, the adjacent rows forming a 
group of rows; | 

examining each group of rows to locate a rowWithin each group having a 
largest number of occurrences relative to other rows Within the respective group; and 

assigning a diagnosis d to each group, the diagnosis d being the position number 
of the bad component for the located row. 
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14. The method oT Claim 13, wherein determining a probability of one or both 
of a correct diagnosis andtan incorrect diagnosis further comprises creating and 
initializing a matrix M such that matrix elements M(iJ) of the matrix M are equal to 
zero for all i and j 9 where i is an integer that ranges from one to m+\ and where j is an 

integer that ranges from one\to m, where m is the number of tests in the test suite. 

V 

\\ 

15. The method of Claim 14, wherein for each group having a test pattern that 
represents no failed tests, determining a probability further comprises: 

adding iteratively for each^row r in the group the number of occurrences value 
of the row r to a current value oftthe matrix element M(m+1, b) to generate a next 
value of the matrix element M(m\l, 6), where b is a position number of the bad 
component for the row r. 



16. The method of Claim 15^ wherein for each group having a test pattern that 
represents at least one failed test, determining a probability further comprises: 

adding iteratively for each row^ in the group the number of occurrences value 
of the row r to a current value of the matrix element M(d 9 b) to generate a next value 
of the matrix element M(d, b). ^ 

17. The method of Claim 16, wherein the determined probability of a correct 
diagnosis P cor r is calculated using \ 

P JL JJE 

where £ is a sum of diagonal elements M(i 9 i) of the matrix M for i equals one to 
m and E is a sum of all number of occurrence values in the database copy. 

18. The method of Claim 17, wherein^using the determined probability to 
evaluate the test suite comprises: 

suggesting a test to add to the test suite to\improve diagnostic efficacy. 



19. The method of Claim 18, wherein suggesting a test comprises: 
finding a relatively largest value element M{iJ) in the matrix M, where i is not 
equal to j 9 the element M(iJ) representing the probability of incorrectly diagnosing 
component i as the bad component when component j is actually bad; and 
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suggesting a test t having high coverage for component i and one of either low 
coverage for component j, i£j is not equal to m+1, or coverage being irrelevant, if / is 
equal to ra+1. 



20. The method of Claim 19, wherein suggesting a test further comprises 
creating a list of suggested tests, wherein creating a list comprises: 

repeating finding and suggesting for each element of a set of largest value 
elements M(i,j), a test being suggested for each element of the set of elements M(iJ)\ 
and 

computing a score for each df the suggested tests, the score being computed by 
dividing the element value M{iJ) by the total accumulated number of occurrences E. 

21. The method of Claim 20, ^/herein the list of suggested tests is represented 
in one or both of human readable form or machine-readable form. 

\ 

22. The method of Claim 17, wherein using the determined probability to 
evaluate the test suite comprises: 

identifying a test t of the test suite that may be deleted from the test suite. 

23. The method of Claim 22, wherein for identifying a test t to delete from the 
test suite, the method further comprises: 

determining a probability of a correct diagnosis P CO rr,t for a modified test suite 
using the database, the modified test suite having a selected test t removed from the 
test suite; 

computing an efficacy value for the modified test suite using the determined 
probabilities for the test suite and for the modified test suite; and 

generating a list of deletable tests, the deletable tests having a lowest associated 
efficacy relative to efficacies of other tests in the t^t suite. 

24. The method of Claim 23, wherein determining a probability of a correct 
diagnosis P CO rr,t for a modified test suite associated vitfith each of the tests t in the test 
suite comprises using a modified database created fro|{n the database copy, wherein 
the modified database is created comprising: 

copying the database copy into another database ^ppy; 
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selecting a test t to remove from the test suite; 

deleting a position frW each test pattern associated with the selected test t from 
the other database copy; an\ 

copying rows of the otner database copy into a modified database, such that any 
rows that have identical values for the component pattern and the test pattern are 
combined together in the modified database, 

wherein in each row of tile modified database that represents a set of combined 
rows from the other database ccray the number of occurrences is a sum of the number 
of occurrence values for the combined rows. 

25. The method of Claim M, wherein the probability of a correct diagnosis 
Pcorr.t for each of the modified test suites is determined in a manner analogous to 
determining the probability of a correct diagnosis P corr for the test suite. 

26. The method of Claim 24, wherein determining a probability of a correct 
diagnosis P CO rr,t for the modified test suite J 1 using the modified database comprises: 

summing a largest number of occurrences value v max found for each unique test 
pattern value within the modified database; and 

dividing the v max sum by a total number of occurrences E h where the total 
number of occurrences E t is the sum of alllnumbers of occurrences in the modified 
database. 

27. The method of Claim 23, whereiA\computing an efficacy value for each of 
the tests in the test suite comprises computing|a difference between the determined 
probability of a correct diagnosis P CO rr,t for thettnodified test suite corresponding to a 
selected test t and the determined probability oflfa correct diagnosis P corr for the test 
suite. 



28. The method of Claim 27, wherein the Wnputed efficacy e(t) value further 
comprises a cost metric c(t) associated with the tesjU, where a(t) = c{t) {P CO r r> t - Pcorr). 



29. The method of Claim 27, wherein the generated list of deletable tests 
comprises an associated efficacy value for each of tnm deletable tests. 
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30. The method M Claim 23, wherein the generated list is represented in one 
or both of human readable form and in machine-readable form. 

3 1 . The method of Claim 1 1 , wherein the created simulation database 
comprises a Monte Carol simulation of the device under test model, the database 
having a set of entries, each entry having a field for a number-of-occurrences value, a 
field for a test result pattern, and a field for a component state pattern. 



32. A system that detenmnes efficacy of a test suite of a model-based 
diagnostic testing system comprising: 
a processor; 
a memory; and 

a computer program stored inVthe memory and executed by the processor, 
wherein the computer program comprises instructions that, when executed by the 
processor, implement evaluating the test suite using a probability of a diagnosis to 
determine the efficacy. 



33. The system of Claim 32, wherein the instructions that evaluate the test 
suite comprise one or both of suggesting a test to add to the test suite, and identifying 
a test to delete from the test suite. 



34. The system of Claim 32, wherein^the instructions that evaluate the test 
suite comprise: 

creating a simulation database of the tesftsuite; 
determining a probability of one or both of a correct diagnosis and an incorrect 
diagnosis using the database; and 

using the determined probability to evaluate the test suite. 

35. The system of Claim 34, wherein using|he determined probability of both 
a correct diagnosis and an incorrect diagnosis compitises creating a list of suggested 
tests to add to the test suite, each suggested test haviftg an associated test coverage. 



36. The system of Claim 34, wherein the instructions that evaluate the test 
suite further comprise: \ 
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determining a probabilit^of a correct diagnosis for a modified test suite using 
the database, the modified test aiite having a selected test removed from the test suite; 

and wherein using the determined probability comprises: 

computing an efficacy valu^for the modified test suite using the determined 
probability of a correct diagnosis wr both the test suite and the modified test suite; 
and 1 

generating a list of tests to delete from the test suite based on the computed 
efficacy value. 

37. The system of Claim 36, wherein determining a probability of a correct 
diagnosis for a modified test suite is reseated for different modified test suites, each 
different modified test suite having an associated different selected test being 
removed. \ 

38. The system of Claim 33, wherein suggesting a test to add to the test suite 
and identifying a test to delete from the testreuite each comprise a list of respective 
tests, the lists being represented in one or bom of human readable form or machine- 
re^adable^form. \ 
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